Variational Autoencoders with Euclidean and Hyperbolic Latent Spaces for Population Genetics

Population structure inference is one of the main problems of population genetics. Genetic variation might give a clue on relations between populations as well as to identify population components in a single individual. Currently, principle component analysis (PCA) is one of standard tools for genetic data structure visualisation. In this work we present the application of variational autoencoders (VAE) with Euclidean and hyperbolic latent spaces and compare these approaches with PCA. In contrast to the PCA, VAE allows to find nonlinear dependencies in the data, and hyperbolic geometry is better suited for data with hierarchical structure. We show that VAEs have more power to separate population components in some complicated population scenarios.